Control apparatus, control method, and program

ABSTRACT

There is provided a control apparatus including: a processing unit configured to decide, on a basis of object information indicating a state of an object, a plurality of imaging devices that image the objects such that each of all the objects included in a predetermined area imaged by the plurality of imaging devices falls within an angle of view at which the at least two or more imaging devices perform imaging, and control each of the plurality of imaging devices.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 15/777,801, which is a U.S. National Phase ofInternational Patent Application No. PCT/JP2016/077004 filed on Sep. 13,2016, which claims priority benefit of Japanese Patent Application No.JP 2015-235467 filed in the Japan Patent Office on Dec. 2, 2015. Each ofthe above-referenced applications is hereby incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates to a control apparatus, a control method,and a program.

BACKGROUND ART

The technology has been developed that tracks and images an object bycontrolling two imaging devices. Examples of the technology thatcontrols, on the basis of captured images acquired from the two imagingdevices equipped with pan heads, the pan heads to track an objectinclude the technology described in Patent Literature 1 below.

CITATION LIST Patent Literature

Patent Literature 1: JP 2003-97914A

DISCLOSURE OF INVENTION Technical Problem

For example, for various purposes of analyzing games of sports,preventing crimes, improving the safety of mobile objects such as cars,or the like, captured images (moving images or still images) taken byeach of a plurality of imaging devices are used.

To achieve the various purposes as described above, it is considered tobe important for the plurality of imaging devices to image all thepredetermined objects included in an area (which will be referred to as“predetermined area” below in some cases) to be imaged by the imagingdevices. Examples of the above-described predetermined area include a“court or grounds (example in the case of the purpose of analyzing gamesof sports),” a “space in which crime preventing measures are taken(example in the case of the purpose of preventing crimes),” a “space inthe advancing direction of a mobile object (example in the case ofimproving the safety of a mobile object),” or the like. In addition,examples of the above-described predetermined objects include “one orboth of a player and a ball (example in the case of the purpose foranalyzing games of sports),” a “person (example in the case of thepurpose for preventing crimes),” “any object (example in the case of thepurpose for improving the safety of a mobile object),” or the like.

Here, examples of a method for imaging all the predetermined objectsincluded in the predetermined area include a method that uses aplurality of imaging devices (which will be referred to as “solid-stateimaging devices” below) having the imaging positions and imagingdirections fixed.

However, in the case where the existing method that uses theabove-described solid-state imaging devices is used, the imagingposition and imaging direction of each of the solid-state imagingdevices is fixed, and thus it is not necessarily possible for all thesolid-state imaging devices to image the predetermined objects.Therefore, in the case where the existing method that uses theabove-described solid-state imaging devices is used, it can beimpossible to efficiently image the predetermined objects.

The present disclosure proposes a novel and improved control apparatus,control method, and program that enable a plurality of imaging devicesto more efficiently image objects included in a predetermined area.

Solution to Problem

According to the present disclosure, there is provided a controlapparatus including: a processing unit configured to decide, on a basisof object information indicating a state of an object, a plurality ofimaging devices that image the objects such that each of all the objectsincluded in a predetermined area imaged by the plurality of imagingdevices falls within an angle of view at which the at least two or moreimaging devices perform imaging, and control each of the plurality ofimaging devices.

In addition, according to the present disclosure, there is provided acontrol method that is executed by a control apparatus, the controlmethod including: a step of deciding, on a basis of object informationindicating a state of an object, a plurality of imaging devices thatimage the objects such that each of all the objects included in apredetermined area imaged by the plurality of imaging devices fallswithin an angle of view at which the at least two or more imagingdevices perform imaging, and controlling each of the plurality ofimaging devices.

In addition, according to the present disclosure, there is provided aprogram for a computer to execute: a function of deciding, on a basis ofobject information indicating a state of an object, a plurality ofimaging devices that image the objects such that each of all the objectsincluded in a predetermined area imaged by the plurality of imagingdevices falls within an angle of view at which the at least two or moreimaging devices perform imaging, and controlling each of the pluralityof imaging devices.

Advantageous Effects of Invention

According to the present disclosure, it is possible to cause a pluralityof imaging devices to more efficiently image an object included in apredetermined area.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram illustrating an example of a controlresult of an imaging device according to the present embodiment.

FIG. 2 is an explanatory diagram illustrating an example of a controlresult of the imaging device according to the present embodiment.

FIGS. 3A and 3B are explanatory diagrams for describing an overview ofcontrol processing of a control method according to the presentembodiment.

FIGS. 4A and 4B are explanatory diagrams illustrating an example of acaptured image according to the present embodiment.

FIGS. 5A and 5B are explanatory diagrams illustrating an example ofimaging ranges of a plurality of imaging devices according to thepresent embodiment.

FIG. 6 is an explanatory diagram for describing an example of a functionfor obtaining size of a predetermined object according to the presentembodiment.

FIGS. 7A and 7B are explanatory diagrams for describing an example of away of obtaining a degree of concentration of a predetermined objectaccording to the present embodiment.

FIG. 8 is an explanatory diagram for describing an example of allocationof the imaging device to each predetermined object according to thepresent embodiment.

FIG. 9 is an explanatory diagram for describing an example in which thepredetermined objects according to the present embodiment are grouped.

FIG. 10 is an explanatory diagram for describing an example ofallocation of the imaging device to each group of predetermined objectsaccording to the present embodiment.

FIGS. 11A and 11B are explanatory diagrams for describing an example ofinitial states of a plurality of imaging devices according to thepresent embodiment.

FIG. 12 is a flowchart illustrating an example of processing that usescontrol processing of a control method according to the presentembodiment.

FIG. 13 is a block diagram illustrating an example of a configuration ofa control apparatus according to the present embodiment.

FIG. 14 is an explanatory diagram illustrating an example of a hardwareconfiguration of the control apparatus according to the presentembodiment.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. Notethat, in this specification and the appended drawings, structuralelements that have substantially the same function and structure aredenoted with the same reference numerals, and repeated explanation ofthese structural elements is omitted.

In addition, description will be made below in the following order.1. Control Method according to the Present Embodiment2. Control Apparatus according to the Present Embodiment3. Program according to the Present Embodiment

(Control Method According to the Present Embodiment)

First, a control method according to the present embodiment will bedescribed. The following shows the case where the processing of thecontrol method according to the present embodiment is performed by acontrol apparatus according to the present embodiment as an example.

The following chiefly shows the case where the control method accordingto the present embodiment is applied to control over a plurality ofimaging devices that image a space, for example, a court, grounds, orthe like in which sports are played as an example. In addition, thefollowing shows the case where the above-described sports are soccer asan example. Note that the control method according to the presentembodiment is applicable to control over a plurality of imaging devicesthat image a space in which various sports are played such as “ballgames including tennis, basketball, rugby, and the like” and “non-ballgames including kabaddi and the like.”

Note that an application example of the control method according to thepresent embodiment is not limited to control over a plurality of imagingdevices that image a space in which the above-described sports areplayed. For example, the control method according to the presentembodiment is applicable to control over various imaging devices such ascontrol over a plurality of imaging devices serving as security cameras,and control over a plurality of imaging devices mounted on a mobileobject including a car or the like.

[1] Overview of Control Method According to the Present Embodiment

As described above, in the case where the existing method that uses thesolid-state imaging devices is used, it can be impossible to efficientlyimage the predetermined objects.

In addition, in the case where the existing method that uses solid-stateimaging devices is used, it is not necessarily possible for all thesolid-state imaging devices to image predetermined objects, but, forexample, the following demerits are more conspicuous with increase inthe number of imaging devices.

-   -   Increased cost of overall system    -   More complicated installation and setting of apparatuses such as        imaging devices (e.g., in the case where a predetermined area is        a court or grounds, or the like)    -   Increased load of processing captured images    -   Increased load of managing and recording captured images

Then, the control apparatus according to the present embodiment controlseach of a plurality of imaging devices (control processing) such thateach of all the predetermined objects included in a predetermined areafalls within the angle of view at which at least two or more imagingdevices perform imaging.

The control apparatus according to the present embodiment performs, ascontrol over each of the plurality of imaging devices, for example, oneor both of control over the imaging direction of each of the pluralityof imaging devices, and control over the magnification or reduction ofan object imaged by each of the plurality of imaging devices.

Here, examples of the control over the imaging directions of the imagingdevices include control over one or two or more of the pan directions,tilt directions, and roll directions of the imaging devices. Forexample, the control apparatus according to the present embodimenttransmits, to pan heads physically connected to the imaging devices andeach capable of being movable in the pan direction, the tilt direction,and the roll direction, control signals that control the operation ofthe pan heads, and controls the operations of the pan heads. The controlapparatus according to the present embodiment transmits theabove-described control signals, for example, via a communication unit(described below) or a connected external communication device. Thecontrol apparatus according to the present embodiment controls theoperation of the pan heads to be capable of changing one or two or moreof the pan directions, tilt directions, and roll directions of theimaging devices, thereby controlling the imaging directions of theimaging devices.

In addition, examples of the control over the magnification or reductionof objects in imaging performed by the imaging devices include controlover the zooming of the imaging devices. For example, the controlapparatus according to the present embodiment transmits, to the imagingdevices, control signals that control the operations of the imagingdevices, and controls the zooming operations of the imaging devices tocontrol the magnification or reduction of objects in imaging performedby the imaging devices. The control apparatus according to the presentembodiment transmits the above-described control signals, for example,via a communication unit (described below) or a connected externalcommunication device.

Note that the control over each of the imaging devices according to thepresent embodiment is not limited to the above-described examples.

For example, it is also possible as the control over each of theplurality of imaging devices for the control apparatus according to thepresent embodiment to further perform control over the position of theimaging device.

The position of an imaging device is acquired, for example, on the basisof a detection result of a sensor that is included in the imaging deviceand usable to identify the position, or a sensor that is connected tothe imaging device and usable to identify the position. Here, examplesof the above-described sensors usable to identify positions include oneor two or more of a global navigation satellite system (GNSS) device,and motion sensors such as an acceleration sensor and an angularvelocity sensor.

In addition, the positions of the imaging devices may be identified, forexample, on the basis of captured images (moving images or still images)generated by the imaging devices performing imaging. For example, thecase where the plurality of imaging devices image soccer grounds isshown as an example. It is possible to compute the positions andattitudes of the imaging devices by using lines, intersections, or thelike of the grounds included in captured images. Here, the positions andattitudes of the imaging devices may be computed on the basis of therelationship between the imaging devices (method for performingcomputation by using the structure-from-motion technology), or theposition of each imaging device may be computed by using the fact thatthe shape of the grounds is known (method for performing computation bysolving the Perspective-n-Point problem).

Examples of the control over the position of an imaging device includecontrol over the movement of a mobile device physically connected to theimaging device (in the case where the imaging device does not have afunction of moving), or control over the movement of an imaging devicehaving a function of moving. Examples of the above-described mobiledevice include a mobile object (e.g., train or the like) that moves on apredetermined route such as a rail, a mobile object (e.g., robot or thelike) that moves on any route, or the like. In addition, examples of theimaging device having a function of moving include an unmanned airvehicle called a drone, a mobile object (e.g., robot or the like)including an imaging device that moves on any route, or the like.

For example, the control apparatus according to the present embodimenttransmits a control signal that controls the operation of a mobiledevice to the mobile device to control the movement of the mobiledevice. The control apparatus according to the present embodimenttransmits the above-described control signals, for example, via acommunication unit (described below) or a connected externalcommunication device. The control apparatus according to the presentembodiment controls the movement of a mobile device to be capable ofchanging the position of the mobile device, thereby controlling theposition of the imaging device.

In addition, for example, the control apparatus according to the presentembodiment transmits a control signal that controls the operation of animaging device to the imaging device to control the position of theimaging device. The control apparatus according to the presentembodiment transmits the above-described control signals, for example,via a communication unit (described below) or a connected externalcommunication device.

The control apparatus according to the present embodiment performs, asdescribed above, for example, one or both of control over the imagingdirection of each of the plurality of imaging devices, and control overthe magnification or reduction of an object imaged by each of theplurality of imaging devices. In addition, for example, as describedabove, the control apparatus according to the present embodiment iscapable of further controlling the position of an imaging device. Thatis, an imaging device to be controlled by the control apparatusaccording to the present embodiment can be an imaging device that doesnot have the “imaging direction,” or the “imaging position and imagingdirection” fixed.

FIGS. 1 and 2 are explanatory diagrams each illustrating an example of acontrol result of an imaging device according to the present embodiment.FIG. 1 illustrates an example of a plurality of imaging devices thatimage soccer grounds (example of the predetermined area). FIG. 2illustrates another example of a plurality of imaging devices that imagesoccer grounds. In addition, P1 illustrated in FIG. 2 represents thecenter-of-gravity position of a predetermined object that is imaged by amovable imaging device D1 illustrated in FIG. 2. P2 illustrated in FIG.2 represents the center-of-gravity position of a predetermined objectthat is imaged by a movable imaging device D2 illustrated in FIG. 2.

The control apparatus according to the present embodiment controls eachimaging device to change the imaging direction of the imaging device,and the magnification and reduction of an object in imaging performed bythe imaging device in accordance with the position of a player or thelike (example of the predetermined objects), for example, as illustratedin FIG. 1. In addition, for example, in the case there is a movableimaging device as illustrated in FIG. 2, the control apparatus accordingto the present embodiment controls each imaging device to change theposition of the movable imaging device in accordance with the positionof a player or the like (example of the predetermined objects). Here, inthe case where the grounds are imaged from right above like the imagingdevices D1 and D2 illustrated in FIG. 2, it is possible to furtherdecrease the possibility of occlusion.

Therefore, the control apparatus according to the present embodimentcontrols each imaging device to enable all the imaging devices to imagethe predetermined objects.

Thus, the control apparatus according to the present embodiment makes itpossible by performing the control processing of the control methodaccording to the present embodiment to cause a plurality of imagingdevices to image objects included in the predetermined area moreefficiently than in the case where the existing method that usessolid-state imaging devices is used.

In addition, the control apparatus according to the present embodimentperforms the control processing of the control method according to thepresent embodiment, and can thus dynamically change the imaging range ofeach imaging device. Therefore, in the case where the control processingof the control method according to the present embodiment is used, it ispossible to image the predetermined objects included in thepredetermined area with less imaging devices than in the case where theexisting method that uses solid-state imaging devices is used.

Thus, in the case where the control processing of the control methodaccording to the present embodiment is used, it is possible to furtherdecrease the above-described demerits that can be observed in the casewhere the existing method which uses solid-state imaging devices isused.

Note that the control processing according to the present embodiment isobtained by dividing the processing of the control method according tothe present embodiment for the sake of convenience. Therefore, it isalso possible to interpret that, as the processing of the control methodaccording to the present embodiment, for example, the control processingaccording to the present embodiment includes two or more kinds ofprocessing (depending on any methods for division).

The following describes the control processing of the control methodaccording to the present embodiment.

[2] Control Processing of Control Method According to the PresentEmbodiment

[2-1] Overview of Control Processing

First, the overview of the control processing according to the presentembodiment will be described. FIGS. 3A and 3B are explanatory diagramsfor describing the overview of the control processing of the controlmethod according to the present embodiment.

In the case where an object O is imaged by one imaging device asillustrated in FIG. 3A, it is not possible to obtain the position of theobject O from a captured image. In contrast, in the case where theobject O is imaged by two or more imaging devices (FIG. 3B illustratesan example in which three imaging devices perform imaging) asillustrated in FIG. 3B, for example, the use of triangulation makes itpossible to obtain the position of the object O from captured images.

Then, the control apparatus according to the present embodiment decidesa plurality of imaging devices that image a predetermined object, andcontrols each of the plurality of imaging devices. The control apparatusaccording to the present embodiment decides a plurality of imagingdevices that image the predetermined objects, for example, on the basisof object information, such that all the predetermined objects includedin the predetermined area fall within the angle of view at which atleast two or more imaging devices perform imaging.

Here, the object information according to the present embodiment is dataindicating the state of a predetermined object. The state of thepredetermined object according to the present embodiment will bedescribed below.

The object information according to the present embodiment is generated,for example, on the basis of a captured image acquired from each of aplurality of imaging devices. The processing of generating the objectinformation may be performed by the control apparatus according to thepresent embodiment, or an external apparatus of the control apparatusaccording to the present embodiment such as a server.

For example, the control apparatus according to the present embodiment“generates object information on the basis of a captured image acquiredfrom each of a plurality of imaging devices,” or “transmits atransmission request including an instruction to transmit the objectinformation to an external apparatus such as a server, and receives theobject information from the external apparatus” to acquire the objectinformation. The control apparatus according to the present embodimentuses, for example, a communication unit (described below) included inthe control apparatus according to the present embodiment, or anexternal communication device connected to the control apparatusaccording to the present embodiment to communicate with the externalapparatus. The following generically refers to the above-describedprocessing of generating object information and acquiring the objectinformation, and the above-described processing of transmitting atransmission request and acquiring the object information as“acquisition processing” in some cases.

Note that the method for the control apparatus according to the presentembodiment to acquire object information is not limited to the controlapparatus according to the present embodiment performing acquisitionprocessing. For example, in the case where object information isgenerated by the external apparatus such as a server, it is alsopossible for the control apparatus according to the present embodimentto acquire the object information that the external apparatus takes theinitiative in transmitting by actively receiving the object information.

That is, it is possible for the control apparatus according to thepresent embodiment to perform control processing, for example, on thebasis of object information acquired by performing acquisitionprocessing as the processing of the control method according to thepresent embodiment, and also perform control processing on the basis ofobject information actively acquired from the external apparatus.

In addition, the predetermined area is an area that a plurality ofimaging devices are capable of imaging. The predetermined areacorresponds to an area to be imaged by the imaging devices as describedabove.

As an example, the case will be shown where the control method accordingto the present embodiment is applied to control over a plurality ofimaging devices that image a space in which sports are played. Examplesof the predetermined area include a space such as a court or grounds inwhich sports are played. In addition, in the case where the controlmethod according to the present embodiment is applied to control over aplurality of imaging devices that image a space in which sports areplayed, the predetermined area may be, for example, the whole of an areasuch as a court or grounds, or a part of an area such as a court orgrounds. Note that, needless to say, examples of the predetermined areaaccording to the present embodiment are not limited to theabove-described examples.

In addition, a predetermined object is an object to be detected amongobjects included in a captured image obtained by imaging thepredetermined area.

The control apparatus according to the present embodiment uses, forexample, the predetermined objects detected from respective capturedimages acquired when “plurality of imaging devices are each caused toenter the initial state in which the whole of the predetermined area isimaged by the plurality of imaging devices” as all the predeterminedobjects according to the present embodiment. In the case where animaging device is caused to enter the initial state, an ID is assignedto each of the predetermined objects detected from the respectivecaptured images. The ID for a predetermined object may be automaticallyassigned to a predetermined object detected according to imageprocessing that uses any object detection technology, or manuallyassigned on the basis of an operation of a user. An example of theinitial state of an imaging device according to the present embodimentwill be described below.

In addition, in the case where object information transmitted from theexternal apparatus such as a server is received, the control apparatusaccording to the present embodiment may use, for example, the respectivepredetermined objects indicated by the object information acquired fromthe external apparatus as all the predetermined objects according to thepresent embodiment. The following chiefly describes the case where allthe predetermined objects according to the present embodiment are thepredetermined objects detected from respective captured images acquiredwhen imaging devices are caused to enter the initial states.

As an example, the case will be shown where the control method accordingto the present embodiment is applied to control over a plurality ofimaging devices that image a space in which sports are played. Examplesof the predetermined objects include one or both of a player and a ball(example of the predetermined objects in the case where ports are ballgames).

Here, in the case where the predetermined objects are a player and aball, the image size of the player included in a captured image islarger than the image size of the ball included in the captured image inmany cases because the ball is smaller than the player. In addition, inthe case where the predetermined objects are a player and a ball, themovement of the ball is faster than the movement of the player in manycases. Therefore, considering processing of tracking the predeterminedobjects (which will be referred to as “tracking processing”), in thecase where the predetermined objects are a player and a ball, it can bemore difficult to track the ball on the basis of a captured image thanto track the player on the basis of the captured image.

The control processing according to the present embodiment is performedto control each of a plurality of imaging devices such that each of allthe predetermined objects falls within the angle of view at which atleast two or more imaging devices perform imaging as described above.Therefore, the control processing according to the present embodiment isperformed, so that it can be easier to track the predetermined objectsin the tracking processing based on captured images.

Note that, needless to say, examples of the predetermined objectsaccording to the present embodiment are not limited to theabove-described examples.

As described above, the control apparatus according to the presentembodiment performs, as control over each of the plurality of imagingdevices, for example, one or both of control over the imaging directionof each of the plurality of imaging devices, and control over themagnification or reduction of an object imaged by each of the pluralityof imaging devices. The control apparatus according to the presentembodiment controls an imaging device, for example, as shown in (a) to(c) below.

(a) Control Over Image Size of Object to be Imaged (Example of ControlOver Magnification or Reduction of Object in Imaging Performed byImaging Device)

FIGS. 4A and 4B are explanatory diagrams illustrating an example of acaptured image according to the present embodiment. FIG. 4A illustratesan example of a captured image obtained by imaging soccer grounds(example of the predetermined area). FIG. 4B illustrates another exampleof a captured image obtained by imaging soccer grounds.

For example, in the case where players (example of the predeterminedobjects: the same applied to the following) included in a captured imageare small as illustrated in FIG. 4A, it can be impossible, for example,to detect any player from the captured image, distinguish the detectedplayers, or the like. Therefore, for example, in the case where theplayers included in the captured image are small as illustrated in FIG.4A, the processing based on a captured image can be disturbed like it isimpossible to track a player on the basis of a captured image, or thelike.

Then, the control apparatus according to the present embodiment controlsthe magnification or reduction of objects in imaging performed by eachof a plurality of imaging devices such that the predetermined objectsincluded in the captured image have certain size or larger when imagedas illustrated in FIG. 4B.

Here, the control apparatus according to the present embodiment controlsthe magnification or reduction of the object in imaging performed byeach of the plurality of imaging devices, for example, such that theimage size of the entire predetermined objects or the image size of partof the predetermined objects is larger than set image size.

For example, in the case where a captured image includes a ball (exampleof the predetermined objects), the control apparatus according to thepresent embodiment controls the magnification or reduction of the objectin imaging performed by each of a plurality of imaging devices such thatthe image size of the ball in the captured image has a size of about20×20 [pixels] or larger. In addition, for example, in the case where acaptured image includes one or two or more players (example of thepredetermined objects), the control apparatus according to the presentembodiment controls the magnification or reduction of the objects inimaging performed by each of a plurality of imaging devices such thatthe image size of the faces of all the players included in the capturedimage or the image size of the numbers assigned to the uniforms of allthe players included in the captured image has a size of about 20×20[pixels] or larger. In addition, for example, in the case where acaptured image includes a ball (example of the predetermined objects)and one or two or more players (example of the predetermined objects),the control apparatus according to the present embodiment controls themagnification or reduction of the objects in imaging performed by eachof a plurality of imaging devices such that the “image size of the ballin the captured image” and the “image size of the faces of all theplayers included in the captured image or the image size of the numbersassigned to the uniforms of all the players included in the capturedimage” have a size of about 20×20 [pixels] or larger.

The control apparatus according to the present embodiment controls themagnification or reduction of the object in imaging performed by each ofthe plurality of imaging devices, for example, as described above, inaccordance with the types of the predetermined objects such that theimage size of the entire predetermined objects or the image size of partof the predetermined objects is larger than set image size.

Here, which of the image size of the entire predetermined objects andthe image size of part of the predetermined objects is set as a controlstandard is based, for example, on whether the predetermined objects areidentified by identifying the entire predetermined objects oridentifying part of the predetermined objects. For example, the aboveexemplifies the “case where players are identified on the basis of thefaces (example of part of the predetermined objects) or the numbers(example of part of the predetermined objects) assigned to the uniform.”

In addition, the above exemplifies a size of about 20×20 [pixels] asimage size in control over the magnification or reduction of an objectin imaging performed by each of a plurality of imaging devices. However,as long as it is possible to identify the predetermined objects, it ispossible to employ any image size.

Note that, needless to say, in the control apparatus according to thepresent embodiment, an example of control over the magnification orreduction of an object in imaging performed by each of a plurality ofimaging devices is not limited to the above-described examples.

(b) Control Over Amount of Change in State of Imaging Device (Examplesof Control Over Imaging Direction of Imaging Device, Control OverMagnification or Reduction of Object in Imaging Performed by ImagingDevice, and Control Over One or Two or More of Positions of ImagingDevices)

It takes time to complete changing the state (one or two or more ofattitude, zoom, and position) of an imaging device. Therefore, forexample, considering sports, the situation of a predetermined objectsuch as a player or a ball like the position of the predeterminedobject, or the like can change before changing the state of the imagingdevice is completed.

In addition, when a small change in the situation of a predeterminedobject in the predetermined area considerably changes the state of animaging device, the state transition of the imaging device can beunstable.

Therefore, the control apparatus according to the present embodiment,for example, provides hysteresis to changes in the state of an imagingdevice, places an upper limit on the amount of change in the state of animaging device, combines them, or the like to control the amount ofchange in the state of the imaging device such that the current state ofthe imaging device smoothly changes.

(c) Control Over Allocation of Imaging Device that Images PredeterminedObject

FIGS. 5A and 5B are explanatory diagrams illustrating an example of theimaging ranges of a plurality of imaging devices according to thepresent embodiment. FIG. 5A illustrates an example of the imaging rangesof a plurality of imaging devices that image soccer grounds (example ofthe predetermined area). FIG. 5B illustrates another example of theimaging ranges of a plurality of imaging devices that image soccergrounds.

As described above, the control apparatus according to the presentembodiment controls each of a plurality of imaging devices such thateach of all the predetermined objects included in a predetermined areafalls within the angle of view at which at least two or more imagingdevices perform imaging.

Therefore, in the case where the predetermined objects unevenly exist inthe predetermined area, for example, as illustrated in FIG. 5A, part ofthe area in which the predetermined objects in the predetermined areaexist is imaged by all the imaging devices, and thus all thepredetermined objects are imaged by at least two or more imagingdevices. At this time, the zoom ratios (angles of view) of the imagingdevices may be set to be the longest focal distance with which thepredetermined objects imaged by the imaging devices are imaged as anexception to the control shown in (a) above.

In addition, in the case where the predetermined objects exist lessunevenly in the predetermined area, for example, as illustrated in FIG.5B, each of the predetermined objects is imaged by at least two or moreimaging devices that have shorter distance to the predetermined object.

[2-2] Specific Example of Control Processing

Next, the control processing of the control method according to thepresent embodiment will be described more specifically.

As described above, the control apparatus according to the presentembodiment controls each of a plurality of imaging devices such thateach of all the predetermined objects included in a predetermined areafalls within the angle of view at which at least two or more imagingdevices perform imaging. The control apparatus according to the presentembodiment performs, for example, any of the processing in (1) below tothe processing shown in (3) below to control each of the plurality ofimaging devices.

(1) First Example of Control Processing

The control apparatus according to the present embodiment decides, onthe basis of one or both of the states of the predetermined objectsindicated by object information and changes in the states of imagingdevices, the plurality of imaging devices that image the respectivepredetermined objects, and controls each of the plurality of imagingdevices.

The state of a predetermined object according the present embodiment isdetected on the basis of a captured image. Here, examples of the stateof a predetermined object according to the present embodiment includeone or both of the size of the predetermined object and the degree ofconcentration of the predetermined object.

The size of a predetermined object according to the present embodimentis obtained, for example, on the basis of the image size of thepredetermined object in a captured image.

FIG. 6 is an explanatory diagram for describing an example of a functionfor obtaining the size of a predetermined object according to thepresent embodiment. In FIG. 6, “object size” shown on the horizontalaxis corresponds to the image size of a predetermined object. Inaddition, in FIG. 6, “S_(ij)” shown on the vertical axis corresponds tothe size of a predetermined object that is denoted with a number j (jrepresents a positive integer) and imaged by an imaging device denotedwith a number i (i represents a positive integer).

The control apparatus according to the present embodiment uses afunction, for example, as illustrated in FIG. 6 to obtain the size of apredetermined object which corresponds to the image size of thepredetermined object.

Note that the method for obtaining the size of a predetermined objectaccording to the present embodiment is not limited to theabove-described example.

For example, the control apparatus according to the present embodimentis capable of obtaining, on the basis of the distance between an imagingdevice and a predetermined object, the size of the predetermined object.The control apparatus according to the present embodiment uses, forexample, the function of the reciprocal of the distance between theimaging device and the predetermined object, and the size of thepredetermined object to obtain the size of the predetermined objectwhich corresponds to the reciprocal of the distance between the imagingdevice and the predetermined object. In addition, the control apparatusaccording to the present embodiment is also capable of obtaining thesize of the predetermined object which corresponds to the distancebetween the imaging device and the predetermined object, for example, byreferring to a table (or a database) in which the distance is associatedwith the size.

The degree of concentration of a predetermined object according to thepresent embodiment is obtained, for example, on the basis of thedistance between one predetermined object and the other predeterminedobjects in a captured image.

FIGS. 7A and 7B are explanatory diagrams for describing an example of away of obtaining the degree of concentration of a predetermined objectaccording to the present embodiment. FIG. 7A illustrates an example ofthe degree of concentration of a predetermined object. FIG. 7Aillustrates an example of states in which the concentration degree islow, and the concentration degree is high. In addition, FIG. 7Billustrates an example of a function for obtaining the degree ofconcentration of a predetermined object. In FIG. 7B, the “total distancebetween objects” shown on the horizontal axis corresponds to the totaldistance between one predetermined object and the other predeterminedobjects in a captured image. In addition, in FIG. 7B, “D_(ij)” shown onthe vertical axis corresponds to the concentration degree of thepredetermined object that is denoted with the number j and imaged by theimaging device denoted with the number i.

Here, the total distance between objects which is illustrated in FIG. 7Bmay be the total distance between the predetermined Hey hey hey heyhello object denoted with the number j and all the other predeterminedobjects included in the predetermined area. In the case where the totaldistance between objects which is illustrated in FIG. 7B is set as the“total distance between the predetermined object denoted with the numberj and all the other predetermined objects included in the predeterminedarea,” the distance between the predetermined object denoted with thenumber j and predetermined objects (i.e., predetermined objects that arenot included in a captured image) outside the angle of view of theimaging device denoted with the number i is considered to be “0” (zero).

In addition, a change in the state of the imaging device is representedas the amount of change which shows a change in the state of the imagingdevice. The amount of change which shows a change in the state of theimaging device is obtained by obtaining a weighting linear sum for eachelement, for example, on the basis of the respective amounts of changein the position of the imaging device, the attitude (roll, pitch, andyaw) of the imaging device, and the zooming (focal distance) of theimaging device.

More specifically, the control apparatus according to the presentembodiment allocates an imaging device to each predetermined object suchthat each of all the predetermined objects falls within the angle ofview at which at least two or more imaging devices perform imaging.

FIG. 8 is an explanatory diagram for describing an example of theallocation of the imaging device to each predetermined object accordingto the present embodiment. FIG. 8 illustrates an example in which thepredetermined objects are players and a ball. In addition, in FIG. 8, acombination of an imaging device and a predetermined object which isprovided with “∘” represents an imaging device allocated to apredetermined object.

When the control apparatus according to the present embodiment allocatesan imaging device to each predetermined object, for example, asillustrated in FIG. 8, the control apparatus according to the presentembodiment computes an evaluation value on the basis of one or both ofthe state of a predetermined object and a change in the state of animaging device.

The control apparatus according to the present embodiment performs, forexample, the calculation shown in the following expression 1 to computean evaluation value F.

Here, the following expression 1 shows an example of calculation in thecase where an evaluation value is computed on the basis of both thestate of a predetermined object and a change in the state of an imagingdevice. “S_(ji)” shown in the expression 1 represents the size of apredetermined object, and “D_(ji)” shown in the expression 1 representsthe concentration degree of a predetermined object. In addition, “T_(j)”shown in the expression 1 represents the amount of change in the stateof an imaging device. In addition, “(W_(S), W_(D), W_(T))” shown in theexpression 1 represent weights. Note that “(W_(S), W_(D), W_(T))” shownin the expression 1 may have different values for respectivepredetermined objects.

[Math. 1]

$\begin{matrix}{F = {\sum\limits_{i = 1}^{j = n}\; \left( {{W_{S} \cdot {\sum\limits_{j = 1}^{j = m}\; S_{ij}}} + {W_{D} \cdot {\sum\limits_{j = 1}^{j = m}\; D_{ij}}} + {W_{T} \cdot T_{i}}} \right)}} & \left( {{Expression}\mspace{14mu} 1} \right)\end{matrix}$

Note that, needless to say, the method for computing an evaluation valueaccording to the present embodiment is not limited to the example shownin the above-described expression 1.

When an evaluation value is computed, the control apparatus according tothe present embodiment decides an imaging device that includes eachpredetermined object within the angle of view on the basis of thecomputed evaluation value.

The control apparatus according to the present embodiment computes anevaluation value for each of a plurality of different allocationpatterns which satisfies, for example, the condition that “each of allthe predetermined objects be imaged by at least two or more imagingdevices.” Then, the control apparatus according to the presentembodiment decides an imaging device that images each predeterminedobject on the basis of the allocation pattern having the highestcomputed evaluation value among the plurality of computed evaluationvalues. That is, the control apparatus according to the presentembodiment decides an imaging device that images each predeterminedobject such that a computed evaluation value is the highest.

Note that the method for deciding an imaging device that images eachpredetermined object on the basis of an evaluation value is not limitedto the above-described example.

For example, in the case where a computed evaluation value is greaterthan a set threshold (or in the case where an evaluation value isgreater than or equal to the threshold), the control apparatus accordingto the present embodiment may decide an imaging device that images eachpredetermined object with the allocation patter corresponding to theevaluation value. Examples of the above-described threshold for anevaluation value include a fixed threshold set in advance, or a variablethreshold that is changeable on the basis of an operation of a user, orthe like.

In the case where an imaging device that images each predeterminedobject is decided according to the threshold processing as describedabove, it can be possible to decide an imaging device that images eachpredetermined object in a shorter time than time required in the casewhere an imaging device that images each predetermined object is decidedsuch that the above-described computed evaluation value is the highest.

For example, when an imaging device that images each predeterminedobject is decided as described above, the control apparatus according tothe present embodiment controls each of a plurality of imaging devices,and causes each imaging device to image a decided predetermined object.

(2) Second Example of Control Processing

In the control processing according to the above-described firstexample, an example is shown in which an imaging device that images eachpredetermined object is decided by allocating an imaging device to eachpredetermined object. However, the control processing according to thepresent embodiment is not limited to the control processing according tothe above-described first example.

For example, the control apparatus according to the present embodimentgroups (or segments) the predetermined objects. Then, in the case wherethe predetermined objects are grouped, the control apparatus accordingto the present embodiment controls each of a plurality of imagingdevices such that each group of predetermined objects falls within theangle of view at which at least two or more imaging devices performimaging.

FIG. 9 is an explanatory diagram for describing an example in which thepredetermined objects according to the present embodiment are grouped.FIG. 9 illustrates an example in which the predetermined objectsincluded in the predetermined area are classified into two groups: agroup 1; and a group 2.

The control apparatus according to the present embodiment groups thepredetermined objects, for example, on the basis of the distance betweenone predetermined object and the other predetermined objects. In thecase where the distance between the predetermined objects is less than aset threshold (or in the case where the distance between thepredetermined objects is less than or equal to the threshold), thecontrol apparatus according to the present embodiment groups thesepredetermined objects. Examples of the above-described threshold forgrouping include a fixed threshold set in advance, or a variablethreshold that is changeable on the basis of an operation of a user, orthe like. In addition, in the case where the predetermined objects aregrouped according to the method that uses the threshold processing asdescribed above, there may be both groups each including a plurality ofpredetermined objects and groups each including one predeterminedobject.

Note that the method for grouping the predetermined objects according tothe present embodiment is not limited to the method that uses thethreshold processing as described above. The predetermined objects maybe grouped according to any method capable of grouping the predeterminedobjects.

In the case where the predetermined objects are grouped, the controlapparatus according to the present embodiment decides an imaging devicethat includes each group of predetermined objects within the angle ofview on the basis of one or both of the state of a group ofpredetermined objects which is detected on the basis of a capturedimage, and a change in the state of an imaging device, and controls eachof the plurality of imaging devices.

Here, examples of the state of a group of the predetermined objectsaccording to the present embodiment include one or both of the size ofthe group of the predetermined objects and the degree of concentrationof the group of the predetermined objects.

The size of a group of the predetermined objects according to thepresent embodiment is obtained, for example, on the basis of the imagesize of the group of the predetermined objects in a captured image.Similarly to the case where the control apparatus according to thepresent embodiment obtains the size of a predetermined object, forexample, in the control processing according to the above-describedfirst example, the control apparatus according to the present embodimentuses the function as illustrated in FIG. 6, or the like to obtain thesize of a predetermined object which corresponds to the image size of agroup of predetermined objects.

The degree of concentration of a group of predetermined objectsaccording to the present embodiment is obtained, for example, on thebasis of the total concentration degree of the predetermined objectsbelonging to the group, or the like.

More specifically, the control apparatus according to the presentembodiment allocates an imaging device to each predetermined object suchthat each of all the groups of the predetermined objects falls withinthe angle of view at which at least two or more imaging devices performimaging.

FIG. 10 is an explanatory diagram for describing an example of theallocation of the imaging device to each group of the predeterminedobjects according to the present embodiment. In FIG. 10, a combinationof an imaging device and a group of the predetermined objects which isprovided with “o” represents an imaging device allocated to the group ofthe predetermined objects.

When the control apparatus according to the present embodiment allocatesan imaging device to each group of predetermined objects, for example,as illustrated in FIG. 10, the control apparatus according to thepresent embodiment computes an evaluation value on the basis of one orboth of the state of the group of predetermined objects, and a change inthe state of the imaging device.

The control apparatus according to the present embodiment performscalculation (e.g., calculation in which the respective elements in theabove-described expression 1 are substituted with the state of the groupof predetermined objects and a change in the state of the imagingdevice) similar to, for example, the calculation of an evaluation valuein the control processing according to the above-described first exampleto compute an evaluation value.

When an evaluation value is computed, the control apparatus according tothe present embodiment decides an imaging device that images each groupof the predetermined objects on the basis of the computed evaluationvalue.

For example, similarly to the control processing according to theabove-described first example, the control apparatus according to thepresent embodiment decides an imaging device that images each group ofpredetermined objects such that the computed evaluation value is thehighest. In addition, for example, similarly to the control processingaccording to the above-described first example, the control apparatusaccording to the present embodiment is also capable of deciding animaging device that images each group of predetermined objects accordingto threshold processing that uses a threshold for an evaluation value.

For example, when an imaging device that images each group of thepredetermined objects is decided as described above, the controlapparatus according to the present embodiment controls each of aplurality of imaging devices, and causes each imaging device to image adecided group of the predetermined objects.

In control processing according to a second example, processing in whicha predetermined object in the control processing according to the firstexample is substituted with a group of predetermined objects asdescribed above is performed. Therefore, in the case where the controlprocessing according to the second example is performed, the number ofgroups of predetermined objects is less than or equal to the number ofpredetermined objects in the control processing according to the firstexample. Accordingly, it is possible to decrease the processing load ascompared with that of the control processing according to the firstexample.

(3) Third Example of Control Processing

The control apparatus according to the present embodiment is capable ofswitching the control processing according to the above-described firstexample and the control processing according to the above-describedsecond example, for example, on the basis of an operation of a user.

In addition, the control processing according to the above-describedfirst example and the control processing according to theabove-described second example may be dynamically switched on the basisof the processing load of the control apparatus according to the presentembodiment or the like. In the case of switching based on the processingload of the control apparatus according to the present embodiment, thecontrol apparatus according to the present embodiment performs thecontrol processing according to the above-described first example, forexample, in the case where the use ratio of a processor is less than apredetermined threshold (example of the case where it is not determinedthat the processing load is high). In addition, the control apparatusaccording to the present embodiment performs the control processingaccording to the above-described second example, for example, in thecase where the use ratio of the processor is greater than or equal tothe predetermined threshold (example of the case where it is determinedthat the processing load is high).

The control apparatus according to the present embodiment performs, forexample, any of the control processing according to the first exampleshown in (1) above to the control processing according to the thirdexample shown in (3) above to control each of a plurality of imagingdevices.

Note that the control processing according to the present embodiment isnot limited to the above-described example.

For example, when a variety of cases such as the case where the controlmethod according to the present embodiment is applied to control over aplurality of imaging devices that image a space in which sports areplayed to which the control method according to the present embodimentis applicable are considered, the positions of a predetermined objectssuch as a player and a ball can change. Therefore, even if a pluralityof imaging devices are temporarily controlled according to the controlmethod according to the present embodiment such that each of all thepredetermined objects falls within the angle of view at which at leasttwo or more imaging devices perform imaging, the movement of thepredetermined objects or the like can make it unknown where any of thepredetermined objects is positioned in the predetermined area. That is,even if a plurality of imaging devices are temporarily controlled suchthat each of all the predetermined objects falls within the angle ofview at which at least two or more imaging devices perform imaging, itcan be impossible to track the predetermined objects according to thetracking processing based on captured images.

Then, for example, in the “case where it is determined that the positionof at least one predetermined object among all the predetermined objectsis not detected from a captured image acquired from each of a pluralityof imaging devices,” the control apparatus according to the presentembodiment causes each of the plurality of imaging devices to enter theinitial state.

For example, when a predetermined object whose position is impossible tobe identified on the basis of a captured image acquired from each of aplurality of imaging devices is detected, the control apparatusaccording to the present embodiment determines that the position of atleast one predetermined object among all the predetermined objects isnot detected.

In addition, for example, when the state in which the position of atleast one predetermined object is not detected continues for a setpredetermined time or more (or when the state in which the position ofat least one predetermined object is not detected continues beyond thepredetermined time: the same applies to the following), the controlapparatus according to the present embodiment may determine that theposition of at least one predetermined object among all thepredetermined objects is not detected. Here, examples of theabove-described predetermined time include 10 [seconds], 20 [seconds],or the like. In addition, the above-described predetermined time may bea fixed time set in advance or a variable time that is changeable on thebasis of an operation of a user, or the like.

As described above, providing a predetermined time from the occurrenceof the state in which the position of at least one predetermined objectis not detected to the determination that the position of at least onepredetermined object among all the predetermined objects is not detectedmakes it possible to prevent an imaging device from frequently enteringthe initial state. Therefore, providing a predetermined time from theoccurrence of the state in which the position of at least onepredetermined object is not detected to the determination that theposition of at least one predetermined object among all thepredetermined objects is not detected makes it possible to improve therobustness.

In addition, until a predetermined time elapses from the occurrence ofthe state in which the position of at least one predetermined object isnot detected, the control apparatus according to the present embodimentmay perform any of the control processing according to the first exampleshown in (1) above to the control processing according to the thirdexample shown in (3) above, for example, in the state in which apredetermined object whose position is impossible to be identified isexcluded.

FIGS. 11A and 11B are explanatory diagrams for describing an example ofthe initial states of a plurality of imaging devices according to thepresent embodiment. FIG. 11A illustrates an example of the state inwhich each of all the predetermined objects is imaged by at least two ormore imaging devices. In addition, FIG. 11B illustrates an example ofthe state in which each of a plurality of imaging devices is caused toenter the initial state.

As illustrated in FIG. 11B, in the initial state, the angle of view ofeach imaging device is set in a wide-angle state (wide-end state), and aplurality of imaging devices image the whole of the grounds (example ofthe predetermined area). Then, as illustrated in FIG. 11B, the controlapparatus according to the present embodiment causes the plurality ofimaging devices to enter the initial states to control each of theplurality of imaging devices such that each of all the predeterminedobjects detected again falls within the angle of view at which at leasttwo or more imaging devices perform imaging.

[2-3] Example of Processing that Uses Control Processing

Next, processing that uses the control processing of the control methodaccording to the present embodiment will be described.

FIG. 12 is a flowchart illustrating an example of processing that usescontrol processing of the control method according to the presentembodiment. FIG. 12 illustrates an example of tracking processing thatuses the control processing. FIG. 12 illustrates, as an example, thecase where the control method according to the present embodiment isapplied to control over a plurality of imaging devices that imagesplayers and a ball (example of the predetermined objects) on grounds(example of the predetermined area) where soccer is played. Here, inFIG. 12, for example, the processing in steps S100 to S106, and S114 ofFIG. 12 corresponds to an example of the control processing according tothe present embodiment.

The control apparatus according to the present embodiment causes eachimaging device to enter the initial state (S100). The control apparatusaccording to the present embodiment sets, for example, the angle of viewof each imaging device in the wide-angle state as illustrated in FIG.11B and moves the position of each imaging device to the initialposition.

The control apparatus according to the present embodiment detects apredetermined object from a captured image acquired from each imagingdevice, and assigns an ID to the predetermined object (S102). An ID maybe automatically assigned to a predetermined object detected accordingto image processing that uses any object detection technology asdescribed above, or manually assigned on the basis of an operation of auser.

The control apparatus according to the present embodiment allocates animaging device to a predetermined object (S104). The control apparatusaccording to the present embodiment allocates an imaging device to apredetermined object on the basis of an evaluation value as described,for example, for the control processing according to the first exampleshown in (1) above and the control processing according to the secondexample shown in (2) above.

When, in step S104, an imaging device is allocated to a predeterminedobject, the control apparatus according to the present embodimentadjusts, for example, the attitude or zooming (focal distance) of eachimaging device (S106). In addition, in the case where there is animaging device whose position is changeable, the control apparatusaccording to the present embodiment may further adjust the position ofthe imaging device.

When the processing in step S106 is performed, the control apparatusaccording to the present embodiment performs tracking processing tocause an imaging device to track each predetermined object (S108).

The control apparatus according to the present embodiment computes theposition of each predetermined object, and the position and attitude ofeach imaging device (S110). The position of a predetermined object isobtained, for example, with triangulation on the basis of a capturedimage. In addition, the position and attitude of an imaging device areobtained, for example, on the basis of one or both of a detection resultof a sensor included in the imaging device and capable of use toidentify the position, and a processing result of a captured image whichis obtained by using structure-from-motion technology or the like.

In addition, the control apparatus according to the present embodimentmay cause a storage unit (described below) or any recording medium suchas an external recording medium connected to the control apparatusaccording to the present embodiment to record, for example, informationindicating a result computed in step S110. Recording the informationindicating a result computed in step S110 in the recording medium makesvarious kinds of analysis possible, for example, analysis of a game ofsports or the like which uses the position of a predetermined object orthe history of the state of an imaging device.

The control apparatus according to the present embodiment determineswhether to finish the processing (S112). For example, in the case wherea predetermined operation (e.g., pushing a finish button, or the like)of a user to finish the processing is detected, the control apparatusaccording to the present embodiment determines that the processing isfinished. In addition, in the case where a predetermined event isdetected on the basis of a captured image or the like, the controlapparatus according to the present embodiment may determine that theprocessing is finished. Here, examples of the case where it isdetermined in step S112 to finish the processing include the case wherethe first half of a soccer game is finished (example of theabove-described predetermined event), the case where a soccer game isfinished (another example of the above-described predetermined event),or the like.

In the case where it is determined in step S112 to finish theprocessing, the control apparatus according to the present embodimentfinishes the processing illustrated in FIG. 12.

In addition, in the case where it is not determined in step S112 tofinish the processing, the control apparatus according to the presentembodiment determines whether or not the positions of all the playersand the ball (all the predetermined objects) are grasped (S114). Forexample, in the case where it is determined that the position of atleast one predetermined object among all the predetermined objects isnot detected from a captured image acquired from each of a plurality ofimaging devices, the control apparatus according to the presentembodiment determines that the positions of all the players and the ballare not grasped.

In the case where it is determined in step S114 that the positions ofall the players and the ball are grasped, the control apparatusaccording to the present embodiment repeats the processing from stepS104. In addition, in the case where it is not determined in step S114that the positions of all the players and the ball are grasped, thecontrol apparatus according to the present embodiment repeats theprocessing from step S100.

The control apparatus according to the present embodiment performs, forexample, the processing illustrated in FIG. 12 as the trackingprocessing that uses the control processing. Note that, needless to say,the example of the processing that uses the control processing accordingto the present embodiment is not limited to the example illustrated inFIG. 12.

(Control Apparatus According to the Present Embodiment)

Next, an example of the configuration of a control apparatus accordingto the present embodiment which is capable of performing theabove-described processing of the control method according to thepresent embodiment will be described.

FIG. 13 is a block diagram illustrating an example of the configurationof a control apparatus 100 according to the present embodiment. Thecontrol apparatus 100 includes, for example, a communication unit 102,and a control unit 104.

In addition, the control apparatus 100 may include, for example, a readonly memory (ROM) (not illustrated), a random access memory (RAM) (notillustrated), a storage unit (not illustrated), an operation unit (notillustrated) that is operable by a user, a display unit (notillustrated) that displays various screen on a display screen, and thelike. The control apparatus 100 connects, for example, theabove-described components to each other by using a bus serving as adata transmission path.

The ROM (not illustrated) stores a program and control data such asarithmetic parameter used by the control unit 104. The RAM (notillustrated) temporarily stores a program and the like executed by thecontrol unit 104.

The storage unit (not illustrated) is a storage means included in thecontrol apparatus 100, and stores, for example, data for the controlmethod according to the present embodiment such as a table (or adatabase) in which distance and size are associated, and various kindsof data such as a variety of applications. Here, examples of the storageunit (not illustrated) include a magnetic recording medium such as ahard disk, a non-volatile memory (nonvolatile memory) such as a flashmemory, and the like. In addition, the storage unit (not illustrated)may be attachable to and detachable from the control apparatus 100.

As the operation unit (not illustrated), an operation input devicedescribed below is included. In addition, as the display unit (notillustrated), a display device described below is included.

[Hardware Configuration Example of Control Apparatus 100]

FIG. 14 is an explanatory diagram illustrating an example of thehardware configuration of the control apparatus 100 according to thepresent embodiment. The control apparatus 100 includes, for example, anMPU 150, an ROM 152, an RAM 154, a recording medium 156, an input/outputinterface 158, an operation input device 160, a display device 162, anda communication interface 164. In addition, the control apparatus 100connects, for example, the components to each other by using a bus 166serving as a data transmission path.

The MPU 150 includes, for example, one or two or more processorsincluding arithmetic circuits such as MPUs, various processing circuits,and the like. The MPU 150 functions as the control unit 104 thatcontrols the overall control apparatus 100. In addition, the MPU 150serves, for example, as a processing unit 110 described below in thecontrol apparatus 100. Note that the processing unit 110 may include adedicated (or general-purpose) circuit (e.g., processor or the like thatis a different entity from the MPU 150) capable of implementing theprocessing of the processing unit 110.

The ROM 152 stores a program, control data such as arithmetic parameter,and the like used by the MPU 150. The RAM 154 temporarily stores, forexample, a program and the like executed by the MPU 150.

The recording medium 156 functions as a storage unit (not illustrated),and stores, for example, data for the control method according to thepresent embodiment such as a table in which distance and size areassociated, and various kinds of data such a variety of applications.Here, examples of the recording medium 156 include a magnetic recordingmedium such as a hard disk, and a non-volatile memory such as a flashmemory. In addition, the recording medium 156 may be attachable to anddetachable from the control apparatus 100.

The input/output interface 158 connects, for example, the operationinput device 160 and the display device 162. The operation input device160 functions as an operation unit (not illustrated), and the displaydevice 162 functions as a display unit (not illustrated). Here, examplesof the input/output interface 158 include a universal serial bus (USB)terminal, a digital visual interface (DVI) terminal, an HDMI(high-definition multimedia interface) (registered trademark) terminal,various processing circuits, and the like.

In addition, the operation input device 160 is included, for example, onthe control apparatus 100, and connected to the input/output interface158 in the control apparatus 100. Examples of the operation input device160 include a button, a direction key, a rotary selector such as a jogdial, a combination thereof, or the like.

In addition, the display device 162 is included, for example, on thecontrol apparatus 100, and connected to the input/output interface 158in the control apparatus 100. Examples of the display device 162 includea liquid crystal display, an organic electro-luminescence (EL) display(which is also referred to as organic light emitting diode (OLED)display), and the like.

Note that, needless to say, the input/output interface 158 is capable ofconnecting to an external device such as an external operation inputdevice (e.g., keyboard, mouse, or the like) of the control apparatus100, and an external display device. In addition, the display device 162may be a device, for example, touch panel or the like that is capable ofdisplay and a user operation.

The communication interface 164 is a communication means included in thecontrol apparatus 100, and functions as the communication unit 102 thatperforms wireless or wired communication, for example, with an externalapparatus such as a control target imaging device via a network (ordirectly).

Here, examples of the communication interface 164 include acommunication antenna and a radio frequency (RF) circuit (wirelesscommunication), an IEEE 802.15.1 port and a transmission/receptioncircuit (wireless communication), an IEEE 802.11 port and atransmission/reception circuit (wireless communication), a local areanetwork (LAN) terminal and a transmission/reception circuit (wiredcommunication), or the like. In addition, examples of the networkaccording to the present embodiment include a wired network such as alocal area network (LAN) and a wide area network (WAN), a wirelessnetwork such as a wireless local area network (WLAN), the Internet usinga communication protocol such as TCP/IP (transmission controlprotocol/internet protocol), or the like.

The control apparatus 100 uses the configuration, for example,illustrated in FIG. 14 to perform the processing of the control methodaccording to the present embodiment. Note that the hardwareconfiguration of the control apparatus 100 according to the presentembodiment is not illustrated the configuration illustrated in FIG. 14.

For example, in the case where the control apparatus 100 communicateswith an external apparatus or the like via a connected externalcommunication device, the control apparatus 100 does not have to includethe communication interface 164. In addition, the communicationinterface 164 may be configured to be communicable with one or two ormore external apparatuses or the like in a plurality of communicationschemes.

In addition, the control apparatus 100 may further include, for example,an imaging device serving as an imaging unit (not illustrated).

In the case where the control apparatus 100 includes an imaging device,it is possible in the control apparatus 100 to perform the processing ofthe control method according to the present embodiment, for example, onthe basis of a captured image generated by the imaging device performingimaging. That is, in the case where the control apparatus 100 furtherincludes an imaging device, the control apparatus 100 may also targetthe included imaging device for control according to the controlprocessing.

The imaging device includes, for example, a lens/imaging element and asignal processing circuit. The lens/imaging element includes, forexample, an optical lens, and an image sensor that uses a plurality ofimaging elements such as complementary metal oxide semiconductors(CMOSs). The signal processing circuit includes, for example, anautomatic gain control (AGC) circuit and an analog-to-digital converter(ADC), and converts analog signals generated by the imaging elementsinto digital signals (image data). In addition, the signal processingcircuit performs, for example, various kinds of processing for RAWdevelopment. Further, the signal processing circuit may also performvarious kinds of signal processing, for example, white balancecorrection processing, color correction processing, gamma correctionprocessing, YCbCr conversion processing, edge enhancement processing,and the like.

In addition, the control apparatus 100 can dispense with, for example,one or two or more of the recording medium 156, the operation inputdevice 160, and the display device 162.

In addition, for example, a part or all of the components illustrated inFIG. 14 (or the components according to a modification) may beimplemented by one or two or more ICs.

An example of the configuration of the control apparatus 100 will bedescribed with reference to FIG. 13 again. The communication unit 102 isa communication means included in the control apparatus 100, andperforms wireless or wired communication with an external apparatus suchas a control target imaging device via a network (or directly). Inaddition, the communication of the communication unit 102 is controlled,for example, by the control unit 104.

Here, examples of the communication unit 102 include a communicationantenna and an RF circuit, a LAN terminal and a transmission/receptioncircuit, and the like. However, the configuration of the communicationunit 102 is not limited to the above. For example, the communicationunit 102 can be configured to support any standard that allows forcommunication of a USB terminal, a transmission/reception circuit, andthe like, or configured in any way that it is possible to communicatewith an external apparatus via a network. In addition, the communicationunit 102 may be configured to be communicable with one or two or moreexternal apparatuses or the like in a plurality of communicationschemes.

The control unit 104 includes, for example, an MPU and the like, andserves to control the overall control apparatus 100. In addition, thecontrol unit 104 includes, for example, the processing unit 110, andtakes the initiative in performing the processing of the control methodaccording to the present embodiment.

The processing unit 110 takes the initiative in performing the controlprocessing according to the present embodiment. The processing unit 110controls each of a plurality of imaging devices on the basis of acaptured image acquired from each of the plurality of imaging devicessuch that each of all the predetermined objects falls within the angleof view at which at least two or more imaging devices perform imaging.The processing unit 110 performs, for example, any of the controlprocessing according to the first example shown in (1) above to thecontrol processing according to the third example shown in (3) above tocontrol each of a plurality of imaging devices.

The control unit 104 includes, for example, the processing unit 110,thereby taking the initiative in performing the processing (e.g.,control processing according to the present embodiment) of the controlmethod according to the present embodiment.

The control apparatus 100 uses the configuration, for example,illustrated in FIG. 13 to perform the processing of the control methodaccording to the present embodiment (e.g., the above-described controlprocessing).

Thus, configured, for example, as illustrated in FIG. 13, the controlapparatus 100 is capable of causing a plurality of imaging devices tomore efficiently image objects included in the predetermined area.

In addition, configured, for example, as illustrated in FIG. 13, thecontrol apparatus 100 is capable of attaining the advantageous effectsattained by performing the processing of the control method according tothe present embodiment.

Note that the configuration of the control apparatus according to thepresent embodiment is not limited to the configuration illustrated inFIG. 13.

For example, the control apparatus according to the present embodimentcan include the processing unit 110 illustrated in FIG. 13 as adifferent entity from the control unit 104 (i.e., implemented by anotherprocessing circuit).

In addition, as described above, the above-described control processingis obtained by dividing the processing of the control method accordingto the present embodiment for the sake of convenience. Therefore, theconfiguration for implementing the processing of the control methodaccording to the present embodiment is not limited to the configurationillustrated in FIG. 13. It is possible to employ the configurationcorresponding to the way of diving the processing of the control methodaccording to the present embodiment.

In addition, the control apparatus according to the present embodimentmay include an acquisition unit (not illustrated) that takes theinitiative in performing acquisition processing according to the presentembodiment. In the control apparatus according to the presentembodiment, for example, the control unit 104 may include an acquisitionunit (not illustrated), or the acquisition unit may be included as adifferent entity from the control unit 104.

In addition, in the case where communication is performed with anexternal apparatus via an external communication device that has afunction or configuration similar to that of the communication unit 102,the control apparatus according to the present embodiment may dispensewith the communication unit 102.

In addition, the control apparatus according to the present embodimentmay further include an imaging unit (not illustrated) including animaging device. In the case where the control apparatus according to thepresent embodiment includes an imaging unit (not illustrated), thecontrol apparatus according to the present embodiment is capable oftargeting an imaging device included in the imaging unit (notillustrated) for control according to the control processing.

The above makes description by using the control apparatus as thepresent embodiment. However, the present embodiment is not limitedthereto. The present embodiment is applicable, for example, to variousapparatuses such as a “computer including a personal computer (PC), aserver, and the like,” an “imaging apparatus having the imaging functionof a camera used for broadcasting, a security camera, an onboard camera,or the like,” a “mobile object including an automobile and the like,” a“tablet type apparatus,” and a “communication apparatus including asmartphone and the like” that are capable of performing the processingof the control method according to the present embodiment. In addition,the present embodiment is also applicable, for example, to a processingIC that can be incorporated into the apparatus as described above.

In addition, the control apparatus according to the present embodimentmay be applied, for example, to a processing system based on aconnection to a network (or communication with each apparatus) likecloud computing or the like. Examples of the processing system in whichthe processing of the control method according to the present embodimentis performed include a system in which one apparatus included in theprocessing system performs the control processing according to thepresent embodiment, and another apparatus included in the processingsystem performs other processing such as tracking processing.

(Program According to the Present Embodiment)

A program (e.g., program according to which it is possible to executethe processing of the control method according to the present embodimentsuch as “the above-described control processing,” or “theabove-described acquisition processing and the above-described controlprocessing”) for causing a computer to function as the control apparatusaccording to the present embodiment is executed by a processor or thelike in a computer, thereby making it possible to cause a plurality ofimaging devices to more efficiently image objects included in thepredetermined area.

In addition, a program for causing a computer to function as the controlapparatus according to the present embodiment is executed by a processoror the like in a computer, thereby making it possible to attain theadvantageous effects attained by the processing of the above-describedcontrol method according to the present embodiment.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

For example, the above shows that a program (computer program) forcausing a computer to function as the control apparatus according to thepresent embodiment is provided. However, the present embodiment can alsofurther provide a recording medium having the above-described programstored therein.

The above-described configuration shows an example of the presentembodiment. Needless to say, the above-described configuration belongsto the technical scope of the present disclosure.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

A control apparatus including:

a processing unit configured to decide, on a basis of object informationindicating a state of an object, a plurality of imaging devices thatimage the objects such that each of all the objects included in apredetermined area imaged by the plurality of imaging devices fallswithin an angle of view at which the at least two or more imagingdevices perform imaging, and control each of the plurality of imagingdevices.

(2)

The control apparatus according to (1), in which

the processing unit decides, on a basis of one or both of the state ofthe object indicated by the object information, and a change in a stateof the imaging device, the plurality of imaging devices that image therespective objects, and controls each of the plurality of imagingdevices.

(3)

The control apparatus according to (2), in which

the processing unit allocates the imaging device to each of the objectssuch that each of all the objects included in the predetermined areafalls within the angle of view at which the at least two or more imagingdevices perform imaging, and computes an evaluation value on the basisof one or both of the state of the object and the change in the state ofthe imaging device, and

the processing unit decides, on a basis of the computed evaluationvalue, the imaging device that includes each of the objects within theangle of view.

(4)

The control apparatus according to (3), in which

the processing unit decides the imaging device that includes each of theobjects within the angle of view such that the computed evaluation valueis highest.

(5)

The control apparatus according to any one of (2) to (4), in which

the object information indicates one or both of size of the object and adegree of concentration of the object.

(6)

The control apparatus according to (5), in which

the size of the object is obtained on a basis of image size of theobject in a captured image acquired from the imaging device.

(7)

The control apparatus according to (5), in which

the size of the object is obtained on a basis of distance between theimaging device and the object.

(8)

The control apparatus according to any one of (5) to (7), in which

the degree of concentration of the object is obtained on a basis ofdistance between the one object and the other object.

(9)

The control apparatus according to any one of (1) to (8), in which

the processing unit groups the plurality of objects on a basis ofdistance between the one object and the other object, and

in a case where the objects are grouped, the processing unit controlseach of the plurality of imaging devices such that each of groups of theobjects falls within the angle of view at which the at least two or moreimaging devices perform imaging.

(10)

The control apparatus according to (9), in which

in the case where the objects are grouped, the processing unit decides,on a basis of one or both of a state of a group of the objects which isdetected on a basis of a captured image acquired from the imagingdevice, and a change in a state of the imaging device, the imagingdevice that includes each of the groups of the objects within the angleof view, and controls each of the plurality of imaging devices.

(11)

The control apparatus according to (10), in which

the processing unit allocates the plurality of imaging devices to eachof the groups of the objects such that each of all the groups of theobjects falls within the angle of view at which the at least two or moreimaging devices perform imaging, and computes an evaluation value on thebasis of one or both of the state of the group of the objects, and thechange in the state of the imaging device, and

the processing unit decides, on a basis of the computed evaluationvalue, the imaging device that includes each of the groups of theobjects within the angle of view.

(12)

The control apparatus according to (10) or (11), in which

the state of the group of the objects is one or both of size of thegroup of the objects, and a degree of concentration of the group of theobjects.

(13)

The control apparatus according to any one of (1) to (12), in which

all the objects are the objects acquired when the processing unit causeseach of the plurality of imaging devices to enter an initial state inwhich the plurality of imaging devices image a whole of thepredetermined area, the objects being detected from respective capturedimages.

(14)

The control apparatus according to (13), in which

in a case where it is determined that a position of at least one of allthe objects is not detected from a captured image acquired from each ofthe plurality of imaging devices, the processing unit causes each of theplurality of imaging devices to enter the initial state.

(15)

The control apparatus according to any one of (1) to (14), in which

the processing unit performs, as control over each of the plurality ofimaging devices, one or both of control over an imaging direction ofeach of the plurality of imaging devices, and control over magnificationor reduction of an object in imaging performed by each of the pluralityof the imaging devices.

(16)

The control apparatus according to (15), in which

the processing unit further performs, as the control over each of theplurality of imaging devices, control over a position of the imagingdevice.

(17)

The control apparatus according to any one of (1) to (16), furtherincluding:

an imaging unit including the imaging device.

(18)

The control apparatus according to any one of (1) to (17), including:

an acquisition unit configured to acquire the object information.

(19)

A control method that is executed by a control apparatus, the controlmethod including:

a step of deciding, on a basis of object information indicating a stateof an object, a plurality of imaging devices that image the objects suchthat each of all the objects included in a predetermined area imaged bythe plurality of imaging devices falls within an angle of view at whichthe at least two or more imaging devices perform imaging, andcontrolling each of the plurality of imaging devices.

(20)

A program for a computer to execute:

a function of deciding, on a basis of object information indicating astate of an object, a plurality of imaging devices that image theobjects such that each of all the objects included in a predeterminedarea imaged by the plurality of imaging devices falls within an angle ofview at which the at least two or more imaging devices perform imaging,and controlling each of the plurality of imaging devices.

REFERENCE SIGNS LIST

-   100 control apparatus-   102 communication unit-   104 control unit-   110 processing unit

1. A control apparatus, comprising: a processor configured to: group aplurality of objects in a specification area into at least one group ofobjects; and control a specific group of cameras of a plurality ofcameras such that at least one object in each group of the at least onegroup of objects falls within an angle of view of at least two camerasof the plurality of cameras.
 2. The control apparatus according to claim1, wherein the specific group of cameras comprises the at least twocameras.
 3. The control apparatus according to claim 1, wherein theprocessor is further configured to determine a controlled camera of theplurality of cameras that includes one group of the at least one groupwithin an angle of view of the controlled camera.
 4. The controlapparatus according to claim 3, wherein the determination of thecontrolled camera is based on a degree of concentration of objects ofthe at least one group.
 5. The control apparatus according to claim 1,wherein the processor is further configured to group the plurality ofobjects in the specification area into the at least one group based on adistance between an object of the plurality of objects and each ofremaining objects of the plurality of objects.
 6. The control apparatusaccording to claim 1, wherein the processor is further configured todetermine allocations of the plurality of cameras that image theplurality of objects based on at least one of a state of an object or achange in a state of at least one camera of the plurality of cameras. 7.The control apparatus according to claim 6, wherein the processor isfurther configured to: compute an evaluation value of each of theallocations based on at least one of the state of the object or thechange in the state of the at least one camera of the plurality ofcameras; and determine the specific group of cameras such that thecomputed evaluation value is highest.
 8. The control apparatus accordingto claim 1, wherein the plurality of objects is acquired at a time inwhich the processor is configured to cause each of the plurality ofcameras to enter an initial state in which the plurality of camerascaptures a whole of the specification area, and the plurality of objectsis detected from respective captured images.
 9. The control apparatusaccording to claim 8, wherein, based on a determination that a positionof at least one of the plurality of objects is not detected from acaptured image acquired from each of the plurality of cameras, theprocessor is further configured to cause each camera of the plurality ofcameras to enter the initial state.
 10. The control apparatus accordingto claim 1, wherein the processor is further configured to control atleast one of an imaging direction of each camera of the plurality ofcameras, or one of magnification or reduction of an object of theplurality of objects captured by each camera of the plurality ofcameras.
 11. The control apparatus according to claim 1, wherein theprocessor is further configured to control a position of the specificgroup of cameras.
 12. A control method, comprising: grouping a pluralityof objects in a specification area into at least one group; andcontrolling a specific group of cameras of a plurality of cameras suchthat at least one object in each group of the at least one group ofobjects falls within an angle of view of at least two cameras of theplurality of cameras.
 13. A non-transitory computer-readable medium,having stored thereon, computer-executable instructions that whenexecuted by a processor of a control apparatus, cause the processor toexecute operations, the operations comprising: grouping a plurality ofobjects in a specification area into at least one group; and controllinga specific group of cameras of a plurality of cameras such that at leastone object in each group of the at least one group of objects fallswithin an angle of view of at least two cameras of the plurality ofcameras.